Fast Feature Selection Using Fractal Dimension

Dimensionality curse and dimensionality reduction are two issues that have retained high interest for data mining, machine learning, multimedia indexing, and clustering. We present a fast, scalable algorithm to quickly select the most important attributes (dimensions) for a given set of n-dimensional vectors. In contrast to older methods, our method has the following desirable properties: (a) it does not do rotation of attributes, thus leading to easy interpretation of the resulting attributes; (b) it can spot attributes that have nonlinear correlations; (c) it requires a constant number of passes over the dataset; (d) it gives a good estimate on how many attributes we should keep. The idea is to use the ‘fractal’ dimension of a dataset as a good approximation of its intrinsic dimension, and to drop attributes that do not affect it. We applied our method on real and synthetic datasets, where it gave fast and good results. 1 Introduction and Motivation When managing the increasing volume of data which is generated by the organizations, a question which frequently arises is: “what part of this data is really relevant to be kept?”. Notice that usually the relations of the database have many attributes which are correlated with the others. Attribute selection is a classic goal, as well as battling the “dimensionality curse” [Berchtold_1998] [Pagel_2000]. A careful chosen subset of attributes improves the performance and efficacy of a variety of algorithms. This is particularly true with redundant data, as many datasets can largely be well-approximated in fewer dimensions. This can also be seen as a way to compress data, as only the attributes which maintain the essential characteristics of the data are kept [Fayyad_1998]. In this paper we introduce a novel technique that can discover how many attributes are significant to characterize a dataset. We also present a fast, scalable algorithm to quickly select the most significant attributes of a dataset. In contrast to other methods, such as Singular Value Decomposition (SVD) [Faloutsos_1996], our method has the following desirable properties: (a) it does not rotate attributes, leading to easy interpretation of the resulting attributes; (b) it can spot attributes that have nonlinear and even non-polynomial correlations; (c) it is linear on the number of objects in the dataset;